home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
TPUG - Toronto PET Users Group
/
TPUG Users Group CD
/
TPUG Users Group CD.iso
/
AMIGA
/
(A)TA
/
(A)TAD.ADF
/
FileTransfers
< prev
next >
Wrap
Text File
|
1986-11-06
|
26KB
|
620 lines
*********************************************************************
This article is being presented through the *StarBoard* Journal of
the FlagShip/StarShip, SIGS (Special Interest Groups) on the
Delphi and GEnie telecommunications networks. Permission is
hereby granted to non-profit organizations only to reprint this
article or pass it along electronically as long as proper credit
is given to both the author and the *StarBoard* Journal.
*********************************************************************
AMIGA TECH TIP: BBS FILE TRANSFERS
by Thad Floryan
(THADF on DELPHI)
Most of you start to explore the world beyond your Amiga after
purchasing a modem and a communication program. Many of you become
participants in the forums and discussion groups on the BBS systems you
reach while finding new "pen pals". Some of you attempt to build your
software libraries from the vast collection of free, public-domain
programs found on the BBS systems, all too often with disappointing
results.
The solution to your BBS woes is understanding how files and programs
are stored and transferred, and knowing how and when to use a special
"toolkit" of programs to fix problems.
----------------------------------------------------------------------
THE BASICS: TEXT FILE FORMATS
Simple text files created with an Amiga editor are stored one character
after another. Each line on an Amiga text file is ended by a line-feed
whose ASCII code is 10 decimal (hex 0A). Other systems end their lines
differently; for example, a Macintosh (tm) file has its lines ended by
a carriage-return whose ASCII code is 13 decimal (hex 0D), whereas an
IBM PC and many mainframes end their lines with both a carriage-return
and a line-feed.
The Amiga's file size will be the number of characters entered; there
is no end of file character. On many other systems the presence of the
character control-Z, whose ASCII code is 26 decimal (hex 1A), indicates
the end of file.
Amiga Notepad text files have their lines also ended by a line-feed,
but there are more characters and binary information after the last
line. The first extra character is a control-Z, and the remainder
describe the pen and paper colors and the font and its style.
The various word processors for the Amiga each have their own file
formats, in which are stored tab settings, margin information, text
rendition, and other specific data. Many of the word processors have a
command to store text as a "simple" file.
Source code statements for many of the programming languages available
on the Amiga are also stored as "simple" ASCII files. For example,
filenames ending with ".c" are usually C programs. The ".c" is the
file's extension, and many other common source file extensions you may
see on the Amiga include:
extension Language
.ASM Assembler
.C C
.FOR Fortran
.MOD Modula-2
.PAS Pascal
.SCR Forth screen file
----------------------------------------------------------------------
THE BASICS: PROGRAM FILE FORMATS
Contrasted with text files, program files are binary and are not easily
digested by editors or word processors. A program file is the
translation of your source program into a machine executable form. As
such, a program file is rigidly formatted into sections called "hunks"
which contain code, data, and other information that control the
loading of your program from disk into memory for execution.
AmigaDOS insists on an exact file format for a program file, else it
will display the dreaded "Unable to load program: file is not an object
module". The most-often cause of this problem is extra data in the file
beyond the "hunkend" section; several cures for this ailment are
described later.
For the technically-minded, the detailed description of the program
file format is in the AmigaDOS Technical Reference Manual (CBM
327266-01) and reprinted in The AmigaDOS Manual (Bantam, ISBN
0-553-34294-0) which should be consulted for specifics.
Program files typically have no extensions, but occasionally one may be
found with the extension ".exe" which is short for "executable". An
extension on a program file is a carryover from other systems and is
not encouraged on the Amiga, especially for programs designed to be
started from the Workbench.
----------------------------------------------------------------------
THE BASICS: DATA AND BINARY FILE FORMATS
Data and binary files are, for the most part, formatted according to
the whims of the creator. There is one notable exception: an
Interchange File Format (IFF) data file for the exchange of pictures,
text and sounds between paint programs, data base systems, information
retrieval systems, and other creative and productivity application
programs. Another exception is the ICON file which may be associated
with a program file or a data file.
The format of an IFF file is described in the documents available on
the Amiga Freely Redistributable Softare Library Disk Number 16. The
disks in this series are colloquially known as "Fish Disks" as tribute
to Fred Fish who has contributed his expertise and time helping make
the Amiga the success it deserves to be. "Fish Disks" are available
from FAUG, many cooperating Amiga Dealers, and by direct subscription.
A program file named MYPROG will usually have an accompanying ICON file
named MYPROG.info. A paint program saving a picture as MYMASTERPIECE
will create a MYMASTERPIECE.info ICON file . A word processor will
create LETTER.info as a companion to a LETTER text data file.
Another binary file type is the object file, which is an intermediate
stage between a source program file and an executable program. Some
software developers allow you to use the fruits of their labors with an
object file while choosing not to reveal how they did it in a source
program file. Object files have "hunk" sections and are meant to be
linked with your other object files to become an executable program.
The file extensions of object files are typically ".o", ".obj", ".sub"
and ".lib".
Data, object and binary files must be exchanged exactly like program
files. Extra or spurious information in such a file will confuse the
recipient or application program. Methods to assure exact transfers
are discussed later.
----------------------------------------------------------------------
THE BASICS: PROTOCOLS
Most BBS systems you reach will either be those operating a version of
BBS-PC software on an Amiga, or those operating a FIDO node on an IBM
PC or variant. All of these systems will offer a Files section for
their validated members. The Files section on these systems will
contain program and data files that you can use on your Amiga.
Copying a file from a BBS system to your Amiga can be almost as easy as
using the COPY command to duplicate files from one place to another on
your system. However, your system and the BBS must speak the same
"language" to transfer files, much as you need to converse in a common
language to exchange thoughts with someone from another country.
In the parlance of the BBS community, the common language is a
"protocol". A protocol is an agreed-upon ritual of greetings,
exchange, and farewells. The most common protocols you'll see have the
names TEXT, XMODEM, XMODEM-CRC, and HVP/Smartcom.
Without getting involved in a history lesson, suffice it to say that
your communication program will understand one or more of these
protocols. You must tell the BBS system which protocol you are using
before you can exchange files successfully.
The TEXT protocol is one in which the contents of the file are
considered to be ASCII characters, and the text of the file is captured
by your terminal program and stored verbatim into a file while you are
reading it. But life is not always so simple. Some files may have
been created on another type of computer system and may have different
characters at the end of each line causing the text to overprint or
display other strange characteristics. Don't despair, a solution to
fix the files on your system is described later.
The XMODEM protocol attempts to transfer the contents of any file to
your system in chunks of 128 characters at a time. Each chunk is
preceded and followed by control information which helps to assure
exact transfer of the data. If the transfer of a chunk of data is
inaccurate, the sender is instructed by the receiver to send it again.
This "send and acknowledge" procedure is called handshaking. Files
transferred using the XMODEM protocol will arrive at the destination
exactly as sent, except the size of the resultant file will be a
multiple of 128 characters or bytes. This "padding" of the received
file is the source of problems transferring Amiga program files;
remember, AmigaDOS expects a program file to adhere to an exact and
specific format. Solutions to this dilemma are discussed later.
A variation of the XMODEM protocol is one in which text files are
processed differently from binary files. This variation will convert,
say, a file stored with carriage-returns and line-feeds, or with
carriage-returns only, to the line-feed only format expected on the
Amiga. Consult the manual for your terminal program to see if this
variation is supported by the program. If not, you may have to
post-process the file after it is downloaded to your Amiga; more
details later.
The XMODEM-CRC protocol is similar to the XMODEM protocol, except that
some additional information is appended to each chunk transferred that
permits the receiver to correct any transmission errors. This
additional "smarts" can save time when transferring files over a noisy
telephone line, because the errors can be corrected locally (on your
system) without requiring the sender to re-transmit the entire chunk
that was in error. The same problems with 128-byte multiple file sizes
apply here as with the standard XMODEM.
The HVP/Smartcom protocol eliminates most of the problems occurring
with XMODEM transfers in that the resultant file will be the exact size
as the source file, and the transmitted chunks are 512 bytes between
handshakes resulting in a faster file transfer. However, if the
original file was uploaded to the BBS with XMODEM, you may still see
the "file is not an object module" when you attempt to run a program
downloaded with HVP/Smartcom. The same solutions for XMODEM
transferred files can be applied to HVP/Smartcom transferred files if
necessary.
----------------------------------------------------------------------
THE CURES: PREPARATION
Many BBS systems provide a file directory listing showing a file's size
and comments posted by the person who originally uploaded the file.
Write down as much information about the file that you can find. One
specific item to seek is any comment referring to a "chop" size; you
may need to examine all E-MAIL postings to find this information.
Knowing whether a file is text, a program, or data will also be
helpful.
----------------------------------------------------------------------
THE CURES: SOFTWARE TOOLS
You will need, as a minimum, copies of the "toolkit" programs listed
below. I suggest you get copies of the following programs direct from
your dealer, users' group, or a friend; some are available on the "Fish
Disks".
USQ Unsqueeze utilty. Its companion is SQ which is used
if you're uploading files to a BBS.
RemCr Removes carriage-returns from a text file. (Leaving,
hopefully, line-feeds as a line terminator).
AddCr Adds carriage-returns before every line-feed. Used
before uploading a file to a BBS.
13to10 Converts all carriage-returns to line-feeds.
Strip Removes control-characters and other binary "junk"
from a text file.
FixObj Adjusts a program file's size so "hunkend" is at the
proper end of file. There is a new version: FixObj2.
CHOP Fixes a file to a previously-known correct size.
ARC Amiga archive utility. Groups collections of files,
and splits them into correct-sized files.
Additionally, the TYPE program (available at the CLI command level on
your Amiga) can be a valuable aid to diagnosing problems with
transferred files.
----------------------------------------------------------------------
PROGRAM TOOLS: TYPE
The TYPE program/command is a starting point to help you determine
which other tools you'll need to correct problems with downloaded
files. With TYPE you can identify executable programs, IFF pictures,
compressed files, text files, and other pertinent characteristics.
For example, suppose you've downloaded a file named XYZ and you want to
find out what it is. The following samples show how to interpret the
hex dump from TYPE. Note that the "***BREAK" is a result of typing a
control-C to interrupt TYPE; after a few lines of output, there is
usually no need to see more.
a)CLI> type XYZ opt h
0000: 000003F3 00000000 00000046 00000000 ...........F....
***BREAK
In this example, XYZ looks like an executable program. The "key" is
the "000003F3" which is the code for a "hunkheader". In the absence
of a file extension, it would be reasonable to assume this file is a
program that could be "fixed" using FixObj or CHOP.
b)CLI> type XYZ opt h
0000: 000003E7 00000002 73747269 70000000 ........strip...
***BREAK
In this example, XYZ looks like an object file (the output of a
language compiler); the clues are "000003E7" (code for a "hunkunit")
and the name "strip". In fact, this file is the object output of the
Lattice C compiler for the STRIP program. Size problems with this file
could be fixed using FixObj or CHOP.
c)CLI> type XYZ opt h
0000: 76FF3BF1 5665726D 6565722D 4769726C v.;.Vermeer-Girl
0010: 00000102 00010004 00030006 00050008 ................
0020: 00070009 0000FF0B 000A000C 006FFFFF .............o..
0030: FF0D000F 000E0011 00100013 00120015 ................
0040: 00140017 00160019 0018001B 001A001D ................
0050: 001C001F 001E0021 00200023 002200FC .......!. .#."..
***BREAK
This one is "tricky"; additional clues are necessary. From the BBS
listing, this file was described as an IFF picture, yet it cannot be
viewed with DeluxePaint (tm), Aegis Images (tm), GraphiCraft (tm),
SeeILBM, ShowILBM or DPSlide, especially since the hex dump does NOT
look like a typical IFF file dump per:
CLI> type Vermeer-Girl opt h
0000: 464F524D 00009CDC 494C424D 424D4844 FORM....ILBMBMHD
0010: 00000014 014000C8 00000000 05000000 .....@..........
0020: 00000A0B 014000C8 434D4150 00000060 .....@..CMAP...`
0030: F0F0F020 1030C050 50D05030 E06040F0 ... .0.PP.P0.`@.
0040: 8050F090 60F0A070 F09000F0 A000F0B0 .P..`..p........
0050: 00F0C040 F0C080F0 B070E0A0 60E09050 ...@.....p..`..P
***BREAK
Based on experience with other files, and because there is an item
("Vermeer-Girl") which could be a file name, this XYZ file then becomes
a candidate for unsqueezing using the USQ program.
As you use TYPE to examine other files, you'll gain experience
identifying files by their contents. Another example using TYPE shows
extraneous characters in a text file:
CLI> type XYZ
First line
Second line
Last line
CLI> type XYZ opt h
0000: 46697273 74206C69 6E650D0A 5365636F First line..Seco
0010: 6E64206C 696E650D 0A4C6173 74206C69 nd line..Last li
0020: 6E650D0A ne..
Note the "0D0A" combinations which are carriage-returns followed by
line-feeds. The carriage-returns can be removed using either the RemCr
or STRIP programs. A final example shows another troublesome
situation:
CLI> type XYZ
Last linene
CLI> type XYZ opt h
0000: 46697273 74206C69 6E650D53 65636F6E First line.Secon
0010: 64206C69 6E650D4C 61737420 6C696E65 d line.Last line
0020: 0D .
with lines ended by only a carriage-return (hex 0D). When typed to the
screen, each line overprints the previous one resulting in "Last
linene". This problem is fixed by using the 13to10 program, which
converts all carriage-returns, whose ASCII code is 13 decimal, to
line-feeds, whose ASCII code is 10 decimal.
----------------------------------------------------------------------
PROGRAM TOOLS: USQ and SQ
The SQueeze and UNsQueeze utilities are gaining popularity on BBS
systems for the storage and transfer of single files (see ARC for
collections of files). The reason is obvious after examining the
directory listing below:
CLI> list
Vermeer-Girl 40164 rwed Yesterday 23:22:52
Vermeer-Girl.qq 17293 rwed Yesterday 23:22:49
The file without the ".qq" extension is a normal IFF ILBM picture file
created with a paint program. The file with the ".qq" is the same file
after compression using SQ. Besides reducing disk space, a squeezed
file will also download faster, saving you money and time.
Additionally, a program, text or data file is ready for immediate use
after unsqueezing! The squeezed file can be transferred using XMODEM
without any worry over the effects of XMODEM padding since the
unsqueezing process will create an exact image of the original file.
The SQ and USQ programs are distributed in both source and executable
form on Fish Disk Number 10. Their use is very straightforward:
CLI> SQ file1 [ file2 ... filen ]
squeezes file1 [ thru filen ] creating file1.qq [ thru filen.qq ].
Very often you'll see file extensions containing the letter "Q" which
is a clue that SQ was used to create the file and that USQ should be
used to unsqueeze the file. By convention, the following extensions
are in common use:
.QQ (none)for a squeezed IFF picture file
.TQT (.TXT)for a squeezed text file
.DQC (.DOC)for a squeezed documentation file
.EQE (.EXE)for a squeezed executable program file
Note also that the original file name is stored in the squeezed file as
shown in example (c) of the TYPE program.
CLI> USQ file.qq [ file2.qq ... filen.qq ]
will sequentially expand file.qq [ thru filen.qq ] to their original
sizes and names.
****** IMPORTANT NOTE ******
You should ALWAYS first display using the "TYPE file OPT H" command the
first part of any file before using USQ. The reason is that often the
original creater of a squeezed file was careless and may have squeezed
a file using a directory path specification as part of the file name;
USQ will attempt a restore to the same path specification on your
system. Referring to example (c) of the TYPE program, the complete
original file name is stored in the squeezed file; if the creator
squeezed from, say, DF2:FILE, the contents of the squeezed file would
be:
CLI> type FILE.QQ opt h
0000: 76FF810B 58595A3A 46494C45 00110002 v...DF2:FILE....
0010: 00010004 00030006 00050008 0007000A ................
0020: 00090096 FF0B0091 FF9AFF93 FFDFFF0D ................
0030: 00F2FF0F 000E000C 0010008B FF8CFF9E ................
0040: FF90FFAC FF8DFF9B FFB3FFB9 FF9CFFF5 ................
***BREAK
and if your system doesn't have a DF2: the USQ will fail. Also, it is
possible for USQ to overwrite another file on your system whose name is
the same as that stored in the squeezed file. An ounce of prevention
is worth a pound of cure; always CHECK before blindly using the USQ
program.
----------------------------------------------------------------------
PROGRAM TOOLS: RemCr, AddCr, 13to10, STRIP
This collection of four programs, and other similar programs, processes
text files by converting, adding or deleting characters. The source
code of these programs is readily available on many BBS systems and in
public domain libraries; feel free to change the programs to suit your
needs. Most utility programs of this nature are written in C, but you
can write similar programs in any language available to you such as
AmigaBasiC.
RemCr simply removes all carriage-returns from a file. Usage:
CLI> RemCr inputfile outputfile
AddCr inserts a carriage-return ahead of every line-feed in a file and
is used to prepare a file for sending to another type of computer.
Usage:
CLI> AddCr inputfile outputfile
13to10 converts all carriage-returns, whose ASCII code is 13 decimal,
to line-feeds, whose ASCII code is 10 decimal. Usage:
CLI> 13to10 inputfile outputfile
STRIP removes most control characters, all NULLs, and other "garbage"
from a text file, leaving only line-feeds, and all printable ASCII
characters and symbols. A file processed by STRIP will be able to be
edited using ED. Caution: this program also removes TAB characters, so
if tabs are important to you, either use another program, or modify
STRIP's source code to preserve the characters of interest to you.
Usage:
CLI> STRIP inputfile outputfile
----------------------------------------------------------------------
PROGRAM TOOLS: FixObj and FixObj2
The FixObj program will fix most problems caused by downloading an
executable program file using XMODEM. FixObj2 is simply an updated
version of FixObj.
FixObj examines an input file for a final "hunkend" and removes
extraneous data beyond that point in the output file. This is the
program to use when you see "Unable to load program: file is not an
object module". Usage:
CLI> FixObj inputfile outputfile
CAUTION: the inputfile cannot reside on a RAM: disk due to a problem
with the "fseek()" file function in V1.1 AmigaDOS.
The source code file and executable program file of FixObj is available
on Fish Disk Number 10. The source code for FixObj2 is starting to
appear on BBS systems around the country.
----------------------------------------------------------------------
PROGRAM TOOLS: CHOP
This program corrects the size of any file whose size must be some
exact quantity. Very often the "chop size" of a file will be listed on
BBS systems either in the file directory descriptions or in E-MAIL
posted by the person who uploaded the program. CHOP was the first
available program on the Amiga that corrected XMODEM padding problems.
Variations of CHOP are written in C and in Basic. All versions expect
an input file name, an output file name, and a chop length. The "chop
length" is the size you want the output file to be. For example:
CLI> CHOP inputfile outputfile 9274
One version of CHOP written in ABasiC is:
50 input% = 1
60 output% = 2
65 count% = 0
100 input "Input file ";infile$
110 input "Output file ";outfile$
115 input "number of bytes to copy";total%
120 open "i", #input%, infile$
130 open "o", #output%, outfile$
200 if eof(input%) then goto 990
201 if (count% = total%) then goto 990
210 get #input%, char$
220 print #output%, using "&"; char$;
221 count% = count% + 1
500 goto 200
990 close #input%
991 close #output%
999 end
----------------------------------------------------------------------
PROGRAM TOOLS: ARC
ARC, also known as AmigArc, is gaining popularity as the "premier" tool
for file storage on BBS systems and for use on one's own system. ARC
collects and compresses groups of files together in a single file. A
multi-file project can have all its files kept together. And, as with
squeezed files, downloading time is reduced because the files are
compressed. For example:
CLI> list
MEMPROJ.ARC 16640 rwed Yesterday 17:10:57
readme 2894 rwed Yesterday 17:46:40
mem.sch 12356 rwed Yesterday 17:46:29
clock.c 4047 rwed Yesterday 17:46:00
clockset.doc 1205 rwed Yesterday 17:46:07
mechanical 7107 rwed Yesterday 17:46:13
memtest.c 468 rwed Yesterday 17:46:31
parts.lst 1670 rwed Yesterday 17:46:35
clock.sch 3980 rwed Yesterday 17:46:04
mem.doc 5284 rwed yesterday 17:46:19
10 files - 129 blocks used
The file named MEMPROJ.ARC is the single file which contains the other
nine files, as shown by:
CLI> ARC l MEMPROJ.ARC
AmigArc v0.11 TEST version. Please do not use after August 1st,1986.
Please refer any comments, BUGS, etc. to:
Raymond S. Brand
c/o The Lido Hotel (a Fido BBS)
(203) 248-7324
or AMUSE (a Fido BBS)
(212) 269-4879
Name Length Date
============ ======== =========
clock.c 4047 22 Jun 86
clock.sch 3980 22 Jun 86
clockset.doc 1205 22 Jun 86
mechanical 7107 22 Jun 86
mem.doc 5284 22 Jun 86
mem.sch 12356 22 Jun 86
memtest.c 468 22 Jun 86
parts.lst 1670 22 Jun 86
readme 2894 22 Jun 86
==== ========
Total 9 39011
Extensive documentation accompanies ARC. The three most useful
commands to ARC are L (to list the contents of the archive file), X (to
extract files from the archive file), and A (to add files to the
archive file). The simplest command forms are:
CLI> ARC L archive.file
CLI> ARC X archive.file
CLI> ARC A archive.file *
To avoid overwriting other files when extracting from an archive and to
avoid including other files when building or adding to an archive, it
is easiest to operate ARC while in a unique subdirectory or on a
freshly formatted disk.
ARC and its documentation is available from many users' groups and
dealers. Updated and improved versions of ARC are available
periodically; the current version is V0.11.
----------------------------------------------------------------------
PROGRAM TOOLS: Improved Terminal Programs
Last but not least, smarter terminal programs are becoming available
that will ease the burden of deciphering problems with XMODEM file
transfers. Some programs include the capabilities of FixObj, and
others combine the functions of RemCr, AddCr, 13to10 and STRIP while
doing a file transfer.
It is to your benefit to throroughly read the documentation that
accompanies your terminal program, and request new features in updated
versions. Many software producers do listen to their customer's
suggestions with the result that everyone "wins" as products evolve to
solve your requirements.
end